Papers with iterative denoising
A Cheaper and Better Diffusion Language Model with Soft-Masked Noise (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing diffusion models have limitations in modeling discrete data, e.g., languages . we present a novel diffusion model for language modeling inspired by linguistic features in languages based on iterative denoising . |
| Approach: | They propose a method that iteratively denoises and adds corruptions to the textual data through soft-masking to better noise it. |
| Outcome: | The proposed model achieves better generation quality and lower training cost than current models with better performance. |
CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit (2026.acl-long)
Copied to clipboard
| Challenge: | Diffusion large language models generate text through iterative denoising with bidirectional attention, enabling richer contextual dependencies. |
| Approach: | They propose a training-free parallel decoding method that fuses Trace Credit with current logits to boost the confidence of correct but underconfident tokens. |
| Outcome: | The proposed method achieves 5.48 times speedup with +0.48 accuracy on LLaDA-8B and is orthogonal to mainstream inference optimizations. |